Host apparatus for simulating two way connectivity for one way data streams

ABSTRACT

A host device is disclosed. The host device includes a processor and a control mechanism. The processor operates to receive primary and associated data, decode and separate the associated data from the primary data, and cause the associated data to be rendered on a separate hand held device. The processor also operates the control mechanism to control a separate display apparatus to separately receive and render the primary data.

RELATED APPLICATIONS

This is a continuation of application Ser. No. 09/003,095, filed on Jan.6, 1998, now U.S. Pat. No. 6,209,132, which is a continuation-in-part ofapplication Ser. No. 08/490/822, filed on Jun. 15, 1995, now U.S. Pat.No. 5,818,441.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to enhancing one-way broadcast datatransmissions. More particularly, it relates to enhancing the qualityand content of a primary information stream by creating and transmittingassociated data which provides the appearance of an interactiveconnection to secondary sources of information.

2. Background

One of the limitations of nearly all forms of electronic mass media suchas radio, television, audio CD's and video cassettes is that thecommunication of information or data is one-way from provider to theconsumer. The characteristics of one-way sources of data are that thedata is sent sequentially and is fleeing or at best can be recorded forlater playback. And a consumer cannot interact with the data provided toseek additional information or services.

There is a desire by electronic mass media consumers for additionalservices, and provider of television and radio broadcast services arelooking for new sources of revenue. In particular, consumers are lookingfor information or data that is related to what they are viewing in themedia. The additional information may be in the form of details on thecontent of the current program such as the recipe of a meal beingdemonstrated on a cooking show or biographies of actors in a drama orhistorical background information on events depicted in a program. Or,it could be program highlights such as key plays from the baseball game.In addition, consumers would like access to real time data such as stockprice, updated baseball scores as they occur, traffic and weatherconditions. Also, consumers would like to have access to specialservices associated with product advertising such as information onwhere to buy nationally advertised products—possible with a map to thenearest location, or the ability to instantly receive coupons.

Recently, on-line services such as provided by the World Wide Web of theInternet have become available. Such services provide access to immenseamounts of data on an interactive basis by linking digital computerstogether over sophisticated communications networks. At the same time,the cost of digital computing power is falling rapidly. For example,many homes now have one or more computers. And home computers havedisplays capable of showing television as well as audio capability. Theyalso have memory and computing power.

But while such services are generally available and computers arebecoming more widespread, finding relevant information is an arduoustask that may take hours of searching even for an experienced user.

In addition, these services require a 2-way connection from a consumer'slocation to the on-line network. In the consumer market, prolongedutilization of a single phone line for 2-way digital communications isexpensive.

Up until now, there has been no way for producers of mass marketbroadcast programming to deliver data associated by its relevancy to itssubject matter that could be interactively displayed and manipulated byconsumers on a real time basis. What is meant by real time is that theconsumer receives and has access to the relevant data during the processof program reception. Therefore the data becomes an integral part of theexperience desired by the program producers.

Although received during a program, the information is retained and maybe traversed by the consumer at a later time, as if they were connectedin a 2-way fashion to an on-line service.

PRIOR ART

Perhaps the earliest effort to address the foregoing needs is forprogram content providers to include a telephone number in either theradio of television broadcast. The consumer is invited to dial a numberfor additional information. However, this telephone number is part ofthe primary data stream which is transitory.

A second attempt is the provision of closed caption programming for thehearing impaired on television. This takes the form of written textappearing somewhere on the television screen typically at the bottom andrequires a special decoder on the television. The text is a writtenrendition of the audio portion of the television program. That is, it isa recapitulation of the information supplied by the primary dataprovider and is not stored or saved for later access by the consumer.

Television networks also use a part of the unused bandwidth in the videosignal to send the schedule of programs coming up on that network. Onenetwork is sending a comprehensive program listing in digital formatover an unused section of the NTSC bandwidth. This is called anelectronic program guide (“EPG”). A second network uses a portion of theunused NTSC bandwidth to send digitally encoded stock quotes and thesubject matter of the topics covered in the news broadcast, and extendeddata services (“XDS”) sends the date, time, name of a scheduled program,type of program and how much of the program remains. Cable servicesprovide additional detail about songs being played on subscriber digitalaudio services such as digital music express (“DMX(tm)”); The detailtypically consists of the name of the artist, the name of the song andthe album. A system called Gemstar(tm), provides information in digitalformat that enables consumers to record programs by referencing a numberin a program guide.

Many of the foregoing services are provided on broadcast television onlyover what is called the vertical blanking interval (the “VBI”). The VBIis a portion of the bandwidth defined for broadcast television and inthe case of NTSC, for example, consists of the first 21 of the 525raster lines that define a video frame. SECAM and PAL have a similararrangement. Of the 21 lines, line 21 is divided into two frames of 60hz bandwidth each. The closed caption is provided in the first frame ofline 21.

None of the foregoing allow a consumer to experience apparentinteractively with external data sources. That is, none provides datathat persists for more than a few seconds and none allows the consumerto manipulate directly that data.

SUMMARY OF THE INVENTION

A host device is disclosed. The host device includes a processor and acontrol mechanism. The processor operates to receive primary andassociated data, decode and separate the associated data from theprimary data, and cause the associated data to be rendered on a separatehand held device. The processor also operates the control mechanism tocontrol a separate display apparatus to separately receive and renderthe primary data.

BRIEF DESCRIPTION OF THE DRAWINGS

The preferred embodiments of the invention will now be described inconnection with the drawing wherein:

FIG. 1 is a block diagram of the invention at the highest level ofabstraction.

FIG. 2 is a block diagram showing one embodiment of the data streamscreation and delivery aspects of a system according to the presentinvention.

FIG. 3 is a block diagram showing the basic and some optional elementsfor forming various embodiments of the data streams consumption aspectof the system.

FIG. 4 is a block diagram showing a functional view of the variousembodiments of the data streams consumption aspect of the system.

FIGS. 5a-5 d are block diagrams illustrating various exemplaryembodiments of the data streams consumption aspect of the system.

FIG. 6 is a block diagram illustrating alternate basic and optionalelements for forming various embodiments for the data streamsconsumption aspect of the system.

FIG. 7 is a block diagram illustrating one embodiment of the hand heldassociated data display and request input device of the presentinvention.

FIG. 8 is a block diagram showing a functional view of the variousembodiments of the data streams consumption aspect of the system.

FIGS. 9a-9 d are block diagrams illustrating various alternate exemplaryembodiments of the data stream consumption aspect of the system.

FIGS. 10a-10 b are block diagrams illustrating further alternateexemplary embodiments of the data stream consumption aspect of thesystem.

FIGS. 11a-11 b are block diagrams illustrating further alternateexemplary embodiments of the data stream consumption aspect of thesystem.

DETAILED DESCRIPTION OF THE INVENTION

An electronic mass media provider may be said to deliver a one-waystream of electronic data. This stream of data is herein referred to asa primary stream of data and includes live or pre-recorded informationthat is created by the broadcasting and entertainment industries for thepurpose of delivery to consumers over delivery mediums such as broadcasttelevision, video cassettes, radio and audio CD's. The content of theprimary data streams typically created by production studios for massdistribution to the consumer market. Often a distributor produces all orpart of the programming content. For example, a local television newsstation obtains pre-produced video news and advertising content fromnational providers and inserts locally produced content and advertisingfor broadcast. Equipment used in the television industry are videocameras and video recorders. Typically, the primary data is organizedinto programs. A primary stream of data can be rendered intelligent to aconsumer as either audio or video or a combination of the two.

The term associated data as used herein refers to a stream of datagenerated separately from the primary data but having content that isrelevant to the primary data in general and usually relevant to aparticular program of primary data and is in this sense associated.Associated data is intended to enhance the utility of the primary datastream, but if it is not available, the primary data stream can stand onits own merits. The creation process typically uses commerciallyavailable software and hardware that output industry standard fileformats such as Hypertext Markup Language (HTML) for text and graphiclayout, Graphics Interchange Format (GIF) Joint Photographic ExpertGroup (JPEG) formats for still images, and so forth.

FIG. 1 is a block diagram of the invention at the highest level ofabstraction. Referring now to FIG. 1, data stream creation unit 1performs the function of generating both the primary data stream and theassociated data stream. The two data streams may or may not be merged tocreate a combined stream of primary and associated data. The two datastreams are supplied to non-interactive delivery unit 2 which performsthe function of delivering the two data streams to the consumer. Thenon-interactive delivery unit includes delivery by broadcast, cable or apackaged media such a as cassettes and audio CD's. Indeed,non-interactive delivery is intended to include all one-way electronicdata delivery systems. The primary data stream and secondary data streamare typically transmitted to a consumer over the same delivery medium;however, the invention contemplates situations where this is not thecase. For example, the primary data stream could be delivered bybroadcast television and the associated data could be delivered over ahigh speed digital network, a FM sideband, a direct satellite broadcast,a cable network, a telephone, etc.

When the delivered data reaches consumer 3, it is decoded from itsdelivery medium. That is, if it is sent by broadcast television, theconsumer decodes the signal, separates the primary data from theassociated data, and stores the associated data for future use. Theprimary data is consumed with or without the associated data. However,when consumed with the associated data, the consumer may selectivelyinteract with the associated data, simulating a two-way connectivity forthe one way data streams.

FIG. 2 provides additional system detail for one embodiment of thepresent invention. Referring now to FIG. 2, reference numeral 10 refersto the primary data stream generation sub-system. The output of primarydata stream generation sub-system 10 is typically an analog signal.However, it may be a digital signal as in the case of digital broadcasttelevision. Data channel 12 connects primary data stream generationsub-system 10 and sequencer 14 and data channel 19 connects it to datasynchronizer 20. Data channels 12 and 19 may be initiated by a livevideo feed, or analog or digital tape. Reference numeral 16 refers to anassociated data generation sub-system. The output of associated datageneration sub-module 16 is a digital signal that is supplied overdigital data link 18 to sequencer 14 and over digital data link 17 tosynchronizer 20. The digital data links of the system may be aconventional digital connection such as a serial or parallel or it maybe a network link. Typical connecting media would be twisted pair,co-axial cable, fiber optic cable or a wireless media.

Reference numeral 20 refers to a data synchronizing sub-system whosefunction is to synchronize the primary data stream generated bysub-system 10 with specific associated data. The input to datasynchronizing sub-system 20 is scene information from the primary datastream in the form of time codes and time duration's, and data fromassociated data generator sub-system 16. It creates a so called scriptfor the delivery and display of associated data at specific points intime. For example, data synchronizer 20 creates a script that specifiesthat a detailed data sheet will be delivered to the consumer prior to aspecific television product advertisement, and that the data sheet willbe displayed on the consumer's display when a certain televisionadvertisement starts

Data synchronizer 20 typically includes software which is patternedafter common digital video editors such as Adobe's Premiere, whichplaces parallel video and audio tracks on a standard time line such asSMPTE (Society of Motion Picture and Television Engineers) time code.This allows segments of each track to be manipulated independently butwith reference to a common time line so that segments may later bereconstituted. The software of data synchronizer 20 allows associateddata to be laid out and manipulated on an additional data track. Thesize of particular associated data components is coordinated with theknown bandwidth of the delivery medium. The user of data synchronizer 20typically specifies that a unit of information should be displayed at aspecific point in time. Data synchronizer 20 then calculates the timerequired to transfer the data and inserts a transfer specification intothe script at the appropriate point prior to the display instruction.The output of data synchronizer sub-system 20 is a digital signalrepresenting a script that synchronizes the output of primary datastream sub-system 10 and associated data stream generator 16.

The output of the data synchronizer 20 is supplied over digital datalink 22 to sequencer sub-system 14. Digital data link 22 is aconventional digital link. Sequencer 14 combines the primary data streamwith the associated data as specified by the script supplied bysynchronizer sub-system 20. Sequencer 14 monitors the output fromprimary data stream sub-system 10 to obtain the information necessary tosequence transmission of associated data. For example, during thebroadcast of a television program, data sequence 14 obtains a time codeinformation such as SMPTE time code which is generated by the primarydata stream. The information is used to coordinate mixing of associateddata with primary data systems. In addition, sequencer 14 mixes otherdata feeds that are being transmitted over the same delivery medium.These may include external data services such as financial data,emergency broadcast information or weather information. Mixing ofexisting data streams is necessary only for delivery mediums that don'tsupport separate, independent transmission of digital information. Thisfunction has the capacity to be used for the delivery of either live orpre-recorded programming, or a combination of both. For liveprogramming, such as a local evening news program, the data streamsequencer takes input directly from the outputs of primary data streamgenerator 10, data synchronizer 20, associated data generator 16, andtime code form the primary data stream. For pre-recorded programming,this function processes the output of the data synchronizer 20 andsequences the broadcast of associated data with the primary data.

The output of sequencer 14 is supplied over conventional digital datalink 24 to encoder 26. Encoder 26 is a hardware component for insertionof digital information into the particular delivery medium that will bereceived by the consumer. If the delivery medium is broadcasttelevision, encoder 26 inserts this information directly into the NTSCtelevision signal, and this delivers both the primary and associateddata over a single delivery medium, broadcast television radio waves.Encoder 26 accepts input streams of both video and serialized digitalinformation. It breaks the serialized digital information up into aseries of packets, and modulates the packets into the (“VBI”) of thevideo signal. Encoder 26 is commercially available hardware andsoftware. A number of vendors market hardware for insertion andextraction of data into and from standard (NTSC/PAL) television signals.Norpak Corporation, EEG, Inc., and WavePhore, Inc. are examples.

The output of encoder 26 is supplied via digital data link 28 totransmitter sub-system 30 which performs the function of physicallytransmitting radio frequency waves into the atmosphere. Suchtransmitters may be television or radio broadcast transmitters or asatellite broadcast system. Also, the data may be stored on a tape forlater transmission. Reference numeral 32 refers to the delivery mediawhich may be radio frequency electromagnetic waves passing through theatmosphere, a video tape, a laser disk, an audio CD and the like. Forthe later media, transmitter 30 becomes a video tape recorder, video oraudio CD recorder respectively.

FIG. 3 illustrates the basic and some optional elements for formingvarious embodiments of the data streams consumption aspect of thesystem. Reference numeral 34 generally indicates these basic andoptional equipment present at the consumer's location. Receiver 36 isused to receive the data streams. Receiver 36 may be an add-in adapterboard, a television or radio broadcast receiver, a cable televisionconverter or a satellite receiver for digital broadcast. . Connected toreceiver 36 is a microprocessor 38 for processing the received datastreams. Connected to microprocessor 38 is memory 40 which is used astemporary storage by microprocessor 38. Memory 40 is typicallysemiconductor RAM. Also connected to microprocessor 38 are humaninterface devices for rendering the primary and associated data, as wellas inputting associated data requests. Human interface devices includeinput device 41 and display device 42. Additionally, optionalnon-volatile storage media 39 such as a hard disk may be provided forcaching received associated data, as well as two way communicationinterface 46 for accessing remote servers may be included in forming thevarious embodiments of the data streams consumption aspect of thesystem. Two way communication interface 46 may be an interface thataccesses the remote server over such media as the analog telephonenetwork, the ISDN digital network, a wide area packet switched networksuch as X25, frame relay or asynchronous transfer mode.

Recall that the delivery media may be radio frequency electromagneticwaves passing through the atmosphere, a video tape, a laser disk, anaudio CD and the like. For the later media, receiver 36 becomes a VCR,laser disk player or an audio CD player respectively.

FIG. 4 is a high level block diagram showing a functional view of thevarious embodiments of system 34 formed with the basic and optionalelements of FIG. 2. Referring now to FIG. 4, receiver 36 receives theprimary and associated data streams from the delivery media, e.g. radiofrequency waves, from input 50. Receiver 36 demodulates the input signaland supplies the primary data stream signal over data path 52 to primarydata rendering sub-system 54 and the associated data signal over datapath 56 to associated data decoder 58.

Primary data rendering sub-system 54 performs the function of presentingthe primary data stream to the consumer in the manner in which a typicalconsumer would expect to see the data presented. For example, in thecase of television, the primary data rendering takes the form of a videoimage typically supplied by a cathode ray tube screen, or possibly aliquid crystal display screen and audio provided by an audio amplifierand speakers. A second example is a broadcast radio demodulator,amplifier and speakers that renders radio waves received via radiobroadcasting audible to a consumer. Yet a third example is a stereosystem that renders data encoded on compact disk or tape audible to theconsumer.

Decoder 58 performs the function of decoding the associated data fromthe delivery media. In one embodiment, decoder 58 consists of an analogto digital converter that converts analog encoded digital data back todigital format.

Associated data protocol manager 60 is connected by one-way data path 62to decode 58 and by one-way data path 64 to communications manager 66.Associated data protocol manager 60 performs the function of extractingthe different forms of associated data from the incoming digital datastream and converting them to a form that can be used by communicationsmanager 66. The types of associated data protocols include World WideWeb pages, closed captioning, stock quotes, sports scores, controlcommands for microprocessor 38 (of FIG. 3) to execute.

Communications manager 66 performs the function of a common networkinterface by receiving data from several different types ofcommunication devices using different data transmission protocols. Suchdevices include telephone modems, ISDN modems, cable modems, wirelessmodems, satellite modems, a broadcast TV, radio and the like.Communication manager 66 converts all data received, regardless of thesource and protocol, into a standard format that can then be utilized bythe rest of the system. This function may be implemented in at least oneof two ways. Proprietary interfaces between the communication componentsmay be designed, or standard industry interfaces such as MicrosoftWindows (a trademark of Microsoft Corporation) sockets may be used. Inone embodiment, Windows sockets are used. A Windows socket is a standardapplication interface to access network data.

Real time trigger 76 is connected to communications manager 66 byone-way data path 78 and to associated data request input driver 88 a bydata path 87. Associated data request input driver 88 a provides theconsumer with the ability to input associated data requests to thesystem. In one embodiment, associated data request input driver 88 ainterfaces to a keyboard and alternate input devices such as a mouse tofacilitate associated data request inputs from a user.

Real time trigger 76 accepts commands sent as part of the associateddata to display a page of information without the user asking for it.The output of real time trigger is a command sent over data path 87 toassociated data request input driver 88 a which causes the page ofinformation to be displayed. For example, a broadcaster may want viewersto see a certain page of information as part of a program that is beingviewed. Real time trigger allows data to be displayed using the samedata protocols as consumer requested information.

Local data storage 80 is connected to communications manager 66 byone-way data path 82 and by data path 83 to local data manager 84. Thehardware implementation for local data storage 80 may be one or more ofthe following; RAM, disk, tape, recordable CD-ROM.

Local data manger 84 is connected to communications manager 66 by datapath 86 and to associated data request input driver 88 a and associateddata rendering driver 88 b by data paths 90 and 94. Local data manager84 receives commands from associated data request input driver 88 a toretrieve associated data from local data storage 80 and sends that datato associated data rendering driver 88 b for presentation to theconsumer. For example, a “Web Browser” may be used to display data pagesfrom the World Wide Web (the “WWW”). Providers of WWW browsers includeNetscape Communications Corp., America Online, Spyglass and others.Local data storage 80 will typically be of limited capacity. Thus, localdata manager 84 purges older and less used information. This isaccomplished by assigning an expiration date and/or a priority toassociated data files. The criteria for determining which data to purgeis settable by the broadcaster and/or the consumer. Criteria includetotal amount of storage available, size of associated data files,expiration date and priority.

Remote data manager 92 (which is included in embodiments provided withthe optional ability to access remote servers) is connected to localdata manager 84, associated data rendering driver 88 b, andcommunications manager 66 by data paths 91, 94 and 96 respectively.Remote data manager 92 receives commands from local data manger 84 toobtain data from remote computers through two-way communications channel74 and send that data to associated data rendering driver 88 b forpresentation to the consumer. Local data manager 84 provides thecommands to remote data manager 92 when it receives requests forassociated data that are not cached in the local storage.

Network protocol manager 68 (which is included in embodiments providedwith the optional ability to access remote servers) is connected to bytwo-way data path 70 to communications manager 66 and by two-way datapath 72 to two-way communication channel 74. Network protocol manager 68performs the function of formatting data received and transmitted overtwo-way communication channel 74. It contains the various protocolsrequired to communicate with remote computers. Again, operating systemssuch as Microsoft's Windows typically provide network protocol managerswith plug in modules called drivers to support different protocols anddifferent communication hardware. Network protocol manager 68 extractsincoming data from two-way communication channel 74 using a networkprotocol appropriate for the medium supported by two-way communicationchannel 74. Network protocol manager 68 also encodes output data usingthe appropriate protocol and passes the data onto two-way communicationchannel 74 for transmission to third party computers.

Two-way communications channel sub-system 74 provides the function forconnecting the client system interactively to remote computers that maybe server machines, machines at online service providers, the Internetor independent Bulletin Board Systems (“BBS”). The network to whichtwo-way communication channel 74 is connected may be a conventionalswitched analog telephone system interfaced to a modem, a digitalswitched system such as ISDN interfaced to an appropriate adapter card,a wide area network connected through an access device, satellitetechnologies, and the like. A two-way communications line is used toretrieve and access information that has only thus far been received bythe client system in the form of references, which are pointers to wherethe information actually resides.

The embodiment illustrated in FIGS. 2-4 operates in the followingmanner. A primary data stream is generated. Associated data isseparately generated. A script is generated that synchronizes how theprimary and associated data are linked together. The primary data, theassociated data and the script are sent to sequencer 14 where they arecombined. The primary data is transmitted in the conventional fashion ofits delivery medium. In the television model, this is broadcast over theair or on cable. The associated data is generally (but not always) sentby the same delivery medium as the primary data. In the case oftelevision, the associated data is encoded in the VBI of the televisionsignal. The signal is received by the consumer's equipment. The primarydata stream is immediately rendered and the associated data is stored inlocal data storage 80. At any time, the consumer may browse the datastored in local data storage. This data will have been professionallyselected to provide an enhanced viewing experience. For example, adifficult to find data sources providing historical background to aprogram will have been previously researched by the associated dataprovider and sent as associated data. The human interface is userfriendly and will allow the consumer to browse through the associateddata. This might start with a menu of available information from whichthe consumer could select just as if the data were coming from an onlineservice. But the data is actually stored locally. And it arrived inlocal storage by being transmitted with the primary data stream. Theconsumer may process the associated data in a variety of ways includingsorting and indexing relevant information.

The addition of a two-way communication channel allows a consumer toalso access online services. In such an embodiment, associated data mayconsist of references such as uniform resource locations (“URL”) whichare WWW page references. Associated data may include references toInternet News Groups which are shared static messages grouped by topicwhich may be accessed. Associated data may also include references toInternet Relay Chat areas which are multi-person dynamic conversationstreams that enable users to communicate by text messages. Or,associated data may include pointers to information on an online servicesuch as America Online, Prodigy or Compuserve. Thus, a menu may listseveral references. And by clicking on a reference, the system actuallyconnects to and retrieves the referenced information from theappropriate source.

Alternatively, the associated data may itself include not only pointersto information on various online services, but the actual informationwhich the associated data provider has acquired from an online serviceor source and incorporated into the associated data prior to itsdelivery to the consumer. For example, the associated data may containactual Internet News Group messages related to the primary data stream.Thus, even if the consumer does not have a two-way communicationschannel or chooses not to use it, the invention provides the appearanceand feel of an interactive connection to remote sources of relevantinformation.

FIGS. 5a-5 d illustrate various exemplary embodiments of the datastreams consumption aspect of the system that can be formed with thebasic and optional elements of FIG. 3. For the exemplary embodimentillustrated in FIG. 5a, receiver 36, microprocessor 38 and memory 40 arecombined to form exemplary computer 35. Exemplary computer 35 is alsoprovided with optional non-volatile storage 39 as well as two waycommunication interface 46. Computer 35 receives the primary andassociated data streams, decodes and separates the data streams, andrenders the primary data with or without the associated data usingdisplay 42, as requested by the user. User input device 43 is used toprovide the associated data request inputs to exemplary computer 35.Exemplary computer 35 is intended to represent a broad category ofgeneral as well as special purpose computers known in the art. Display42 is typically a monitor, whereas user input device 43 is a keyboardwith or without a complementary cursor control device.

FIG. 5b illustrates an alternate embodiment. The difference between theembodiments illustrated in FIGS. 5a and 5 b is the fact that exemplarycomputer 35 is formed without receiver 36, and television 42 withreceiver 36 is used as a display for the primary and associated datainstead. For this embodiment, exemplary computer 35 receives the primaryand associated data streams through receiver 36 of TV 42. Exemplarycomputer 35 decodes and separates the primary and associated datastreams, and renders the primary data with or without the associateddata, as described earlier.

FIG. 5c illustrates yet another alternate embodiment. In FIG. 5c, TVtuner 36, microprocessor 38, memory 40 and CRT 42 are used to formexemplary set top box 35. For the illustrated embodiment, set top box 35is complemented with optional external storage device 39. While notshown, set box 35 may also be provided with optional two waycommunication interface 46. Similar to the earlier embodiments,exemplary set top box 35 receives the primary and associated datastreams, decodes and separates the data streams, and renders the primarydata with or without the associated data using TV 42, as requested bythe user. User input device 43 is used to provide the associated datarequest inputs to exemplary set top box 35.

FIG. 5d illustrates yet another alternate embodiment. In FIG. 5d,receiver 36, microprocessor 38 and memory 40 are used to form exemplaryPC/TV 35 (i.e. a TV with integrated digital computing capability). Forthe illustrated embodiment, exemplary PC/TV 35 is also provided withoptional non-volatile storage device 39 and two way communicationinterface 46. Similar to the earlier embodiments, exemplary PC/TV 35receives the primary and associated data streams, decodes and separatesthe data streams, and renders the primary data with or without theassociated data using CRT 42, as requested by the user. User inputdevice 43 is used to provide the associated data request inputs toexemplary PC/TV 35.

FIG. 6 illustrates an alternate collection of basic and optionalelements for forming the various embodiments of the data streamsconsumption aspect of the system. The difference between the elementsillustrated in FIG. 6 and the elements illustrated in FIG. 3 is the factthat at least two separate display devices, 42 and 200, are employed torender the primary and the associated data. Preferably, associated datadisplay device 200 is an integrated hand held associated display andrequest input interface device. In other words, associated datainterface device 200 is a display as well as an input device, having arelatively small form factor suitable for hand held operation. In oneembodiment, integrated hand held associated data interface device 200 isa wireless device. In other words integrated hand held associated datainterface device 200 communicates with processor 308 through thewireless medium, Additionally, by virtue of the employment of separatedisplay device 200 to render the associated data, primary data displaydevice 42 may be “loosely coupled” to processor 38 or even decoupledfrom processor 38.

FIG. 7 illustrates an exemplary embodiment of the integrated associateddata display and request input interface device. As shown, integratedassociated data display and request input interface device 200 includescommunication interface 202, microprocessor 204, memory 206 and displayelement 208, coupled to each other as shown. These elements are packagedwith enclosure 212 having physical dimensions consistent with hand helddevices known in the art, to allow a user to comfortably operate thedevice while holding the device in his/her hands. For the illustratedembodiment, exemplary integrated associated data display and requestinput interface device 200 is also provided with stylus input 210 toallow a user to interact with a graphical end-user interface to provideassociated data request inputs. Furthermore, communication interface 202is a wireless transmitter/receiver, whereas display element 208 is aflat panel display. Elements 202-210 are intended to represent a broadcategory of these elements known in the art. In particular,microprocessor 204 are intended to represent 8-bit micro-controllers,16-bit DSP processors, as well as 32-bits or greater general purposemicroprocessors.

FIG. 8 illustrates an alternate functional view of data streamconsumption aspect of the system, when a separate display device havingits own processor is employed to render the associated data. FIG. 8differs from FIG. 4 in that local data manager 84 and remote datamanager 92 invoke graphical display interface 106 to render theassociated data. Instead of rendering the associated data on aphysically coupled display device, graphical display interface 106provides the graphical rendering commands to communication manager 66,which in turn transmits them, over the wireless medium, to communicationmanager 222 of the integrated hand held associated data display andrequest input interface device 200.

Over in the integrated associated display and request input interface200, communication manager 222 receives and forwards the graphicalrendering commands to graphical display interface 224, which in turnsexecutes the commands, causing the associated data to be rendered on theassociated data interface device 200.

Additionally, data request input driver 226 forwards any associated datarequest inputs it receives from the user to communication manager 222for transmission, over the wireless medium, to “host” communicationmanager 66. Over in the “host” system, communication manager 66 receivesand forwards the associated data request inputs to local data manager 84through input request driver 104. Local data manager 84 in turnsprocesses the received associated data request inputs as describedearlier.

FIGS. 9a-9 d illustrate various exemplary embodiments of the datastreams consumption aspect of the system that can be formed with thebasic and optional elements of FIG. 6. The exemplary embodiments aresimilar to those of FIGS. 5a-5 d, except for the employment of theintegrated wireless hand held associated data interface 200, and theprovision of wireless transmitter/receiver 102 to the computer, set topbox or PC/TV, to facilitate wireless communication between the “host”system and the wireless hand held associated data interface.

In FIG. 9a, receiver 36, microprocessor 38 etc., together with wirelesstransmitter/receiver 102, are employed to form exemplary computer 35.Exemplary computer 35 first receives the primary and the associateddata, then decodes and separates the associated data from the primarydata, as exemplary computer 35 of FIG. 5a. However unlike the earlierdescribed embodiment, exemplary computer 35 of FIG. 9a renders theprimary data on display 42, and transmits the commands for rendering theassociated data, through the wireless medium, to integrated wirelesshand held associated data interface 200, to cause the associated data tobe rendered.

In FIG. 9b, microprocessor 38 etc., together with wirelesstransmitter/receiver 102, are employed to form exemplary computer 35.Exemplary computer 35 first receives the primary and the associated datausing receiver 36 of TV 42, then decodes and separates the associateddata from the primary data, as exemplary computer of FIG. 5b. Howeverunlike the earlier described embodiment, exemplary computer 35 rendersthe primary data on TV 42, but transmits the commands for rendering theassociated data, through wireless medium, to integrated wireless handheld associated data interface 200, to cause the associated data to berendered.

In FIG. 9c, receiver 36, microprocessor 38 etc., together with wirelesstransmitter/receiver 102, are employed to form exemplary set top box 35.Exemplary set top box 35 first receives the primary and the associateddata using receiver 36 of TV 42, then decodes and separates theassociated data from the primary data, as set top box 35 of FIG. 5c.However unlike the earlier described embodiment, exemplary computer 35renders the primary data on TV 42, but transmits the commands forrendering the associated data, through the wireless medium, tointegrated wireless hand held associated data interface 200, to causethe associated data to be rendered.

In FIG. 9d, TV tuner 36, microprocessor 38 etc., together with wirelesstransmitter/receiver 102, are employed to form exemplary PC/TV 35.Exemplary PC/TV 35 first receives the primary and the associated data,then decodes and separates the associated data from the primary data, asPC/TV 35 of FIG. 5d. However unlike the earlier described embodiment,exemplary PC/TV 35 renders the primary data on CRT 42, but transmits thecommands for rendering the associated data, through the wireless medium,to integrated wirelss hand held associated data interface 200, to causethe associated data to be rendered.

In each of these embodiments, a user may selectively interact andconsume the associated data holding the integrated wireless hand heldassociated data interface 200, while consuming the primary data renderedon a physically decoupled monitor or TV. Therefore, each of theseembodiments advantageously enables a user to comfortably consume theprimary and associated data, e.g. in a family room setting; anadvantageous feature that is unavailable from the embodiments of FIGS.5a-5 d.

As alluded to earlier, with the employment of a separate display deviceto render the associated data, in particular, a wireless hand helddevice, the display device for rendering the primary data may be“loosely” coupled to the processor that processes the associated data.FIGS. 10a-10 b illustrate two further alternate exemplary embodimentsthat can be formed using the alternate collection of elementsillustrated in FIG. 6. In FIG. 10a, TV 42 employed to render the primarydata includes IR receiver 106 for receiving remote control commandsprovided through the infrared spectrum, and exemplary computer 35 issimilarly constituted as exemplary computer 35 of FIG. 9a, as well asprovided with a compatible IR receiver 104. Thus, exemplary computer 35can track the remote control commands provided to TV 42, in particular,channel tuning commands, allowing exemplary computer 35 to control itsown receiver 36 to tune to the same channel to extract, and in turns toprovide the correct associated data, through the wireless medium, towireless hand held associated data interface 200,. Accordingly, TV 42and exemplary computer 35 may be decoupled, further facilitatingcomfortable consumption of the primary and associated data in e.g. afamily room setting.

Likewise, in FIG. 10b, TV 42 employed to render the primary dataincludes IR receiver 106 for receiving remote control commands providedthrough the infrared spectrum, and exemplary set tip box 35 is similarlyconstituted as exemplary set top box 35 of FIG. 9c, as well as providedwith a compatible IR receiver 104. Thus, exemplary set top box 35 cantrack the remote control commands provided to TV 42, in particular,channel tuning commands, allowing exemplary set top box 35 to controlits own receiver 36 to tune to the same channel to extract, and in turnsto provide the correct associated data, through the wireless medium, towireless hand held associated data interface 200. Accordingly, TV 42 andexemplary set top box 35 may be decoupled, further facilitatingcomfortable consumption of the primary and associated data in e.g. afamily room setting.

FIGS. 11a-11 b illustrate two further alternate exemplary embodimentsthat can be formed using the alternate collection of elementsillustrated in FIG. 6. Exemplary computer and set top box 35 of FIGS.11a-11 b are similar to the embodiments of FIGS. 10a-10 b respectively,except each of exemplary computer and set top box 35 of FIGS. 11a-11 bis further provided with IR transmitter 108 to transmit remote controlcommands to control TV 42. Thus, instead of having to provide compatibleIR receivers 104 and 106 to both TV 42 and exemplary computer/set topbox 35, incompatible ones may be provided, as long as exemplarycomputer/set top box 35 is provided with the capability of transmittingremote control commands that are understandable to TV 42. Such abilitycan be provided for example by providing a universal remote controlcommands database to exemplary computer/set top box 35 of FIGS. 11a-11b. In other words, exemplary computer/set up box 35 of FIGS. 11a-11 bmay be advantageously used with a number of existing TV to facilitatecomfortable consumption of the primary and associated data in e.g. afamily room setting.

The foregoing preferred embodiments are subject to numerous adaptationsand modifications without departing from the concept of the invention.Therefore, within the scope of the appended claims, the invention may bepracticed other than as specifically described herein.

What is claimed is:
 1. A system, comprising: a primary data streamgenerator to generate a primary data stream comprising primary data,wherein the primary data comprises broadcast programming; an associateddata stream generator to generate an associated data stream comprisingassociated data, wherein the associated data comprises informationrelevant to the primary data; a data synchronizer for synchronizing theprimary data stream and the associated data stream to generate a scriptbased on a predetermined criteria, wherein the script specifies thepoints in time when the primary data stream and the associated datastream are delivered with respect to each other; a sequencer to combinethe primary data stream with the associated data stream as specified inthe script to sequentially transmit the associated data stream withrespect to the primary data stream; an encoder to encode the primarydata stream and the associated data stream; and a transmitter fortransmitting the primary data stream to a primary data device and theassociated data stream to a remotely-located associated data device. 2.The system of claim 1, wherein the script further specifies the pointsin time when the primary data and the associated data are displayed withrespect to each other.
 3. The system of claim 1, further comprising: areceiver for receiving the primary data stream and the associated datastream; a decoder to decode the primary data stream and the associateddata stream; and a device to render and display the primary data and theassociated data.
 4. The system of claim 1, wherein the script isdelivered to a user before delivering the primary data stream and theassociated data stream to the user.
 5. The system of claim 1, whereinthe associated data stream is interactively accessible.
 6. An apparatus,comprising: a primary data stream generator to generate a primary datastream comprising primary data, wherein the primary data comprisesbroadcast programming; an associated data stream generator to generatean associated data stream comprising associated data, wherein theassociated data comprises information relevant to the primary data; adata synchronizer for synchronizing the primary data stream and theassociated data stream to generate a script based on a predeterminedcriteria, wherein the script specifies the points in time when theprimary data stream and the associated data stream are delivered withrespect to each other; and a sequencer to combine the primary datastream with the associated data stream as specified in the script tosequentially transmit the associated data stream with respect to theprimary data stream.
 7. The apparatus of claim 6, wherein the scriptfurther specifies the points in time when the primary data and theassociated data are displayed with respect to each other.
 8. Theapparatus of claim 6, wherein the script is delivered to a user beforedelivering the primary data stream and the associated data stream to theuser.
 9. The apparatus of claim 6, wherein the associated data stream isinteractively accessible.
 10. The apparatus of claim 6, furthercomprising: an encoder to encode the primary data stream and theassociated data stream; a decoder to decode the primary data stream andthe associated data stream; and a transmitter for transmitting theprimary data stream to a primary data device and the associated datastream to a remotely-located associated data device.
 11. The apparatusof claim 6, further comprising: a receiver for receiving the primarydata stream and the associated data stream; and a device to render anddisplay the primary data and the associated data.
 12. A method,comprising: generating a primary data stream comprising primary data,wherein the primary data comprises broadcast programming; generating anassociated data stream comprising associated data, wherein theassociated data comprises information relevant to the primary data;synchronizing the primary data stream and the associated data stream togenerate a script based on a predetermined criteria, wherein the scriptspecifies the points in time when the primary data stream and theassociated data stream are delivered with respect to each other; andcombining the primary data stream with the associated data stream asspecified in the script to sequentially transmit the associated datastream with respect to the primary data stream.
 13. The method of claim12, wherein the script further specifies the points in time when theprimary data and the associated data are displayed with respect to eachother.
 14. The method of claim 12, further comprising delivering thescript to a user before delivering the primary data stream and theassociated data stream to the user.
 15. The method of claim 12, whereinthe associated data stream is interactively accessible.
 16. The methodof claim 12, further comprising: encoding the primary data stream andthe associated data stream; decoding the primary data stream and theassociated data stream; and transmitting the primary data stream to aprimary data device and the associated data stream to a remotely-locatedassociated data device.
 17. The method of claim 12, further comprising:receiving the primary data stream and the associated data stream; andrendering and displaying the primary data and the associated data.